FEAT: Pagination support for /jobs endpoint#103
Open
prasadlohakpure wants to merge 3 commits intomainfrom
Open
FEAT: Pagination support for /jobs endpoint#103prasadlohakpure wants to merge 3 commits intomainfrom
prasadlohakpure wants to merge 3 commits intomainfrom
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds cursor-based pagination to the Heimdall /jobs listing endpoint to enable incremental fetching of job history while enforcing page-size constraints and returning pagination metadata to clients.
Changes:
- Add
cursorfiltering and moveLIMIThandling into the DAL for dynamic page sizing. - Fetch
pageSize+1rows to determineHasMoreand computeNextCursor. - Extend the shared
resultsetresponse wrapper with pagination fields for JSON responses.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| internal/pkg/heimdall/job_dal.go | Adds cursor filter support, limit extraction/capping, dynamic SQL limit, and populates pagination metadata in responses. |
| internal/pkg/heimdall/queries/job/select_jobs.sql | Removes hardcoded LIMIT to allow DAL-driven pagination. |
| internal/pkg/heimdall/result.go | Extends API response wrapper to include pagination metadata fields in JSON. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
prasadlohakpure
requested review from
Yash Shrivastava (alephys26),
sanketjadhavSF and
Will Graham (wlggraham)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This pull request adds cursor-based pagination to the jobs listing endpoint in the Heimdall package. The main changes include extracting and enforcing page size limits, updating the SQL query to support cursor and limit parameters, and modifying the result set to indicate if more results are available and provide a cursor for the next page.
Pagination and query improvements:
cursorfilter and removed the hardcoded SQLLIMITinselect_jobs.sql, enabling dynamic pagination through the API (internal/pkg/heimdall/job_dal.go,internal/pkg/heimdall/queries/job/select_jobs.sql). [1] [2]defaultPageSizeandmaxPageSize), and updated the query to fetch one extra row to determine if more results exist (internal/pkg/heimdall/job_dal.go). [1] [2]*Added support for filtering using comma seperated tag values
e.g.:
Result structure changes:
resultsetstruct to includeHasMoreandNextCursorfields, and updated the handler to populate these fields based on the query results (internal/pkg/heimdall/result.go,internal/pkg/heimdall/job_dal.go). [1] [2]Dependency updates:
strconvandstringsto support parsing and manipulating filter parameters and queries (internal/pkg/heimdall/job_dal.go).Testing:
Result : Gives 5 objects in respone
Result : Default page size(101) or present number of jobs, whichever is lower along with
"has_more": true/false (depending on values available)
"next_cursor": 112
Result : Gives 5 objects ordered by system_job_id, and ahead of cursor specified
Result : Gives 5 objects ordered by system_job_id, and ahead of cursor specified with exact match for both tags